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(54) Motion measuring device 



(57) An endoscopic apparatus for measuring blood 
flow contained within an enclosed body cavity is 
claimed. Such apparatus comprises an electromagnetic 
radiation source (6) for emitting a coherent beam, an 
optical relay system for propagating the beam down an 
endoscopic probe to illuminate a target (10) within an 
enclosed cavity, an endoscopic optical relay system for 
capturing the reflected light and transmitting an image 
of the illuminated region onto the surface of a two-di- 



mensional image sensor (1), an analog to digital con- 
version device for capturing analog output signals from 
the image sensor for conversion to digital format, a dis- 
play device for immediate visual display of captured im- 
ages, a method for calculating flow values from a cap- 
tured digital image containing speckle structure informa- 
tion of the laser illuminated surface, a memory device 
for storing digital blood flow images, and a display de- 
vice for immediate visual display of color coded, blood 
flow images. 
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Description 

CROSS-REFERENCE TO RELATED APPLICATIONS 

5 [0001] This application claims the benefit of priority under 35 USC 119(e)(1) of provisional patent application no. 
60/342 : 116 filed December 26, 2001 , the content of which is hereby incorporated by reference. 

BACKGROUND OF THE INVENTION 

'0 [0002] This invention relates to a system, method of measurement, and presentation of motion within a cavity, used 
for example to determine the magnitude of blood perfusion to a tissue located within a body cavity. 
[0003] When living tissue is illuminated by laser light the light reflected off the natural surface irregularities interferes 
constructively and destructively producing a random interference pattern called speckle. The speckle pattern produced 
by a stationary object remains static in time and is said to have a high speckle contrast. If the object contains several 

'5 individual particles undergoing motion, such as red blood cells, then the phase difference between the interfering beams 
will change in time. The resulting changes in light intensity reflecting off the tissue can be measured and analyzed to 
produce an index linearly related to tissue blood flow. Using this principle, Stern et. al. developed a technique for 
measuring skin blood flow using a collimated laser beam, pinhole aperture and photomultiplier tube (M. D. Stern, "In 
vivo evaluation of microcirculation by coherent light scattering". Nature, 254, pp 56 - 58, 1975.) 

20 [0004] Imaging techniques for mapping tissue blood flow have subsequently evolved from Stern's work and are 
known in the art as laser speckle methods. Fercher and Briers applied laser speckle using a photographic method for 
capturing speckle contrast images, an optical filtering technique for mapping retinal vascular structure and a region of 
tissue fully illuminated by laser light (A.F Fercher and J.D. Briers, "Flow visualization by means of single-exposure 
photography". Optics Communications. 37:5, 326-330, 1981.) 

25 [0005] More recently Briers described a speckle imaging method that used a CCD camera and electronic processing 
rather than photography and optical filtering techniques. He used this method to generate blood flow maps of skin 
rather than retinal blood flow (J. D. Briers, G. R. Richards, and X. W. He, "Capillary blood flow monitoring using laser 
speckle contrast analysis (LASCA)", J of Biomed. Opt. 4:1, pp. 164-175, 1999;"J. D. Briers, and S. Webster, "Quasi 
real-time digital version of single-exposure speckle photography for full-field monitoring of velocity or flow fields". Opt. 

30 Comm. 116, pp 36-42, 1995). A limiting feature of Brier's technique is that the electronic image processing method for 
determining speckle contrast requires a 7 x 7 pixel window. This window reduces the original speckle images from 512 
X 512 pixel to 73 x 73 pixel resolution, resulting in a very low-resolution blood flow map. Another limiting feature is that 
the Brier contrast method for generating blood flow maps is not a relative measure of blood flow and can only be used 
to display the sharp contrast between areas of high flow embedded within a stationary or (comparably low flow) sur- 

35 rounding tissue matrix. Although speckle contrast images are useful for- displaying tissue vascular structure, they do 
not provide a clinical technique for the immediate and visual display of "true" blood flow maps, where each pixel within 
the image is linearly related to blood flow and several images can be captured for the dynamic measurement of relative 
blood flow and blood flow changes. More recently Dunn and co-workers have used the Briercontrast method to produce 
cerebral blood flow maps (A. K. Dunn, H. Bolay, M. A. Moskowitz, and D. A. Boas, "Dynamic Imaging of Cerebral Blood 

io Flow Using Laser Speckle". Journal of Cerebral Blood Flow and Metabolism 21 , 1 95-201 , 2001) where the CCD camera 
is placed directly above a few millimeters area of surgically exposed cerebral tissue. 

[0006] Another laser speckle technique is presented in US patent number 4862894 and has been specifically applied 
to the measurement of skin blood flow (H. Fujii, et. al. "Evaluation of blood flow by laser speckle image sensing", Applied 
Optics 26:24, pp. 5321- 5325, 1987) and subchondral bone blood flow (S. Fukuoka, T. Hotokebuchi, K. Terada, N. 

45 Kobara, H. Fujii, Y. Sugioka, and Y. Iwamoto, "Assessment of subchondral blood flow in the rabbit femoral condyle 
using the laser speckle method." J. of Orth. Res., 1 7, pp 368-375, 1 999; S. Fukuoka, T. Hotokebuchi, S. Jingushi, H. 
Fujii, Y. Sugioka, and Y. Iwamoto, "Evaluation of blood flow within the subchindral bone of the femoral head: Use of 
the laser speckle method at surgery for osteonecrosis." J. of Orth. Res., 17, pp 80-87. 1999). This instrument uses a 
one-dimensional photo-detector array and a scanning arrangement to produce two-dimensional maps of tissue blood 

so flow rather than a CCD camera. A collimated laser line is projected onto the tissue surface and the resultant line of 
speckles is simultaneously focused onto the linear CCD device. A time differentiated technique for measuring blood 
flow is applied rather than the method of measuring speckle contrast so a very fast CCD readout device with consequent 
low resolution is required. Using the scanning mirror assembly, Fujii and co-workers produce a two-dimensional map 
of tissue blood flow with atypical pixel resolution of 128 X 64 (Fuji, 1987). More recently Fujii has replaced the scanning 

55 mirror sensor with a high-speed, two-dimensional, image sensor (1 00 x1 00 pixels) and modified the above time differ- 
entiated speckle technique into an ophthalmic device for measuring retinal microcirculation (N. Konishi and H. Fujii, 
"Real time visualization of retinal microcirculation by laserflowgraphy". Opt. Eng. 34, No . 4, pp 753-757, 1 995; Y. Tamaki, 
M. Araie, E. Kawamoto, S. Eguchi, and H. Fujii, "Noncontact, Two-dimensional measurement of retinal microcirculation 
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using laser speckle phenomenon". Inv. Opth. And Vis. Sci., 35, No. 11 , pp 3825-3834, 1994). This laser speckle tech- 
nique is described in US patents 51 63437 and 5240006. It is specifically designed for ophthalmic work and small field 
of views related to the size of the eye fundus. 

[0007] No speckle imaging technique described in the cited art is adapted or could be used for endoscopy. A common 

5 feature of all of spockle imaging techniques described in the cited art is that they are remote techniques, which require 
the instrument to be placed directly above or in front of the tissue surface. Remote speckle imaging techniques require 
fixation of the instrument and the tissue surface. During endoscopic speckle imaging, the surgeon and patient may be 
expected to move and cause distortion and artifact in the image. Another common feature of prior art speckle imaging 
techniques is that their image resolution is too low to be useful for clinical endoscopy. Endoscopy is a technique that 

10 requires immediate visual display to assist in cavity navigation and tissue recognition, and serious surgical decisions 
are often made based on what the surgeon "sees". The minimum image resolution required for clinical endoscopy is 
a standard video resolution of 640 x 480 pixels. This resolution is maintained for a variable field of view that can range 
from a few square millimeters to several square centimeters. Prior art speckle-imaging techniques are limited in a low 
resolution and restricted field of view. Therefore, applying prior art speckle techniques to endoscopy would result in 

'5 an image that is of too low of resolution or much too small to be clinically useful. 

[0008] Presently, endoscopic surgery is conducted routinely in hospital operating rooms on a daily basis. Much or- 
thopedic endoscopic surgery is devoted to diagnosis of underlying conditions such as injury, inflammatory arthritis and 
osteoarthritis. In the surgical art diagnostic information is based on tactile and visual inspection of the tissue through 
the endoscope. While this information renders data pertaining to the structure of the tissue under examination, it does 

20 not provide information about the functional integrity of the tissue structures. Those skilled in the surgical arts would 
expect the assessment of the metabolic state of tissue physiology in response to injury or inflammation to provide a 
useful diagnostic. For example such a diagnostic would be useful in determining whether tissue should be repaired or 
resected when the predicted outcome of healing, based on such endoscopic tactile and visual inspection of tissue 
anatomy, is uncertain. 

25 [0009] Those skilled in the surgical art would expect that measuring tissue temperature could be used to assess the 
metabolic state of tissue. For example it is well known that the inflammatory response of tissue to injury results in local 
elevation of tissue temperature. The measurement of tissue temperature in a manner that does not perturb the meas- 
urement is problematic. The use of thermometry to assess the metabolic state of tissue has limited clinical relevance 
because the measurement is local to the point of tissue contact. More recently the methods of thermography have 

30 been used to assess tissue metabolic state, in particular for the diagnosis on malignancy. This method relies upon the 
imaging of mid infrared radiation emitted for tissue. Mid infrared radiation will not transmit through the materials of the 
imaging optics used in endoscopes so the use of thermography in combination with an endoscope is not possible. 



SUMMARY OF THE INVENTION 



[0010] In this patent, we disclose a method for endoscopic tissue diagnosis, which makes use of blood perfusion as 
a secondary indicator of tissue metabolism. The inventors have demonstrated a high degree of correlation between 
blood perfusion and the predicted outcome of healing. There are both high and low thresholds in blood flow, which 
predict the potential for the healing of connective tissues. For example the determination of whether or not injured 
meniscus tissue should be repaired or resected, when the predicted outcome of healing based on tactile and visual 
inspection of tissue anatomy alone is uncertain, may be made using endoscopic imaging of blood perfusion. The 
methods of endoscopic blood perfusion imaging disclosed in this patent may be used as a surgical guide. In the event 
that an inadequate vascular response indicates poor healing, the degree of resection and the margin of resection when 
removing tissue that are avascular and non-viable are determined from the perfusion image. In general, in orthopedic 
surgery, a tissue perfusion image, which shows regions of low metabolic state of tissue physiology, is useful in deter- 
mining conservative surgical resection margins. Conservative surgery may result in improved preservation of mechan- 
ical load bearing function of an injured joint. 

[0011] In the present invention the limitations of prior art speckle imaging are overcome and a method of clinical 
endoscopic speckle imaging of blood flow is disclosed. 

[001 2] An embodiment of an apparatus according to the invention for example comprises a laser arranged to illumi- 
nate a body cavity, a device arranged to provide endoscopic imaging of laser-illuminated tissue and an image analyzer 
and display. Light from the laser source is guided to the tissue surface using a light guide, and laser light exiting this 
light guide produces a substantially uniform and bright illumination of the target tissue to enable effective speckle 
imaging through an endoscope. 

[0013] In one aspect of the invention, speckle from the target tissue is imaged using an endoscope and camera, 
preferably the same endoscope and standard camera that are used for surgery. Image processing methods are applied 
to speckle images, captured by the endoscopic camera, so as to create blood flow images. 

[001 4] Such image processing methods enable fast acquisition of blood flow images to minimize movement artifact 
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that might be normally associated with endoscopic surgery. 

[0015] In another aspect of the invention, such image processing methods are applied to the speckle images so as 
to enable one to one pixel correspondence with the standard color video used for surgery thus permitting identification 
of blood flow in anatomical features and boundaries of the tissue. 
5 [0016] In another aspect of the invention, such image processing methods are applied to speckle images so as to 
enable the generation of blood flow images linearly related to tissue blood flow. 

[0017] In another aspect of the invention, such image processing utilizes averaging techniques and artifact movement 
recognition software to enhance blood flow image quality and eliminate movement artifact that is not associated with 
tissue blood flow. 

10 [0018] These and other aspects of the invention are described in the detailed description and claimed in the claims 
that follow. 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 [0019] There will now be described preferred embodiments of the invention, with reference to the drawings, by way 
of illustration, in which like numerals denote like elements and in which: 

FIG. 1 shows an overall schematic of the manner for imaging tissue blood flow using endoscopic methods; 
FIG. 2 is a detailed schematic of the endoscopic device and preferred embodiment for the endoscope light guide 
20 method of guiding the laser light to the tissue surface; 

FIG. 3 is a detailed schematic of the endoscopic device and alternate embodiment for the endoscope light guide 
method of guiding the laser light to the tissue surface; and 

FIG 4 is a detailed schematic of the method for illuminating the target region and technique for eliminating motion 
artifact. 

25 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

[0020] Comprising is used in the claims in its non-exclusive sense. The indefinite article "a" or "an" before an element 
when used in the claims does not exclude.the possibility that more than one of the element is present. The term light 
30 means all possible frequencies of electro-magnetic radiation having utility for motion detection, as for example within 
a human body, the term laser means all forms of coherent light sources suitable for use in endoscopic work, and the 
term electronic camera means all forms of two-dimensional, electromagnetic radiation receiving arrays suitable for 
receiving light emitted by a laser. 

[0021] Referring to FIG. 1 and FIG 2, an exemplary endoscopic motion measurement device makes use of much of 
35 existing endoscopic hardware consisting of: camera 1, endoscope shaft 2 containing imaging relay lens system 29 
and internal light guide 28, standard object illumination white light source 3, light-guide 4 for delivering white light from 
source to endoscope internal light guide entrance port 9, and video display components 5. 

[0022] For flow or motion measurements using the methods claimed here, sensitivity to low flow and range of meas- 
urement is maximized when the laser source is highly polarized and monochromatic. This is because polarized mon- 

40 ochromatic light sources produce the highest speckle contrast in images of stationary objects. Standard endoscopic 
procedures use non-polarized, white light sources, where polarization and mode preservation of the source light is not 
necessary. Thus, the source light guide 4 and endoscopic internal light guide 28 on standard endoscopes may consist 
of one or several multimode fiber optic cables that do not necessarily preserve the light mode quality. As a result, light 
mode and polarization degrades as a function of traveling distance through these light guides, preventing a proper 

45 blood flow measurement. However, it is favorable to utilize as much of the standard endoscopic equipment as is possible 
because it minimizes modification to the surgical procedure. 

[0023] We have determined that most standard white light source guides 4 consist of a fiber-bundle of several meters 
length that result in too much mode degradation for proper blood flow measurement. However, the endoscopic shaft 
2 containing the internal light guide 28 in most standard endoscopes is of a short enough length that polarization and 
so mode degradation of the source light does not degrade significantly and blood flow measurement is possible. Thus, 
the preferred embodiment of the method for guiding laser light to the distal tip of the endoscope to illuminate tissue of 
interest 10 uses internal light guide 28 contained within endoscope shaft 2. 

[0024] FIG 2 is a detailed schematic demonstrating the method of laser light transmittance to the tissue of interest 
for the preferred embodiment of the instrumentation shown in FIG1. Modification to the standard light delivery hardware 
55 includes a laser light delivery system including: laser transmitter 6 and lens arrangement 7 for coupling laser light to 
fiber optic cable 8 for delivering laser light to the endoscope internal light guide 28, and optical or mechanical switching 
device 30 for switching optical connection of laser source fiber 8 and white light source fiber 4 to endoscope light guide 
entrance port 9. 



4 



EP 1 324 051 A1 



[0025] An alternate embodiment of the method of laser light delivery to the distal tip of the endoscope is given in FIG 
3. The laser light output from the transmitters is focused into a fiber optic cable 8 using an optical arrangement 7. The 
fiber optic cable delivers the laser light to the endoscope system by fitting down the smaller barrel of a double-barreled, 
steel sleeve 31 . The endoscope shaft 2 containing relay lens arrangement 29 slides into the larger barrel of sleeve 31 

s such that the objective Ions of the endoscopic relay lens system 29 has access to the tissue of interest 1 0. The smaller 
barrel of sleeve 31 is used for structure and for guiding fiber optic cable 8 to the distal tip of the endoscope system, 
maintaining a similar optical axis between the laser light delivery system and the relay lens light capture system of the 
standard endoscope hardware. The distal tip of the smaller fiber optic barrel of sleeve 31 contains an optical arrange- 
ment 23 comprising of lenses and prisms designed to expand the diverging laser light exiting the fiber and to correct 

10 for any deviations in the optical axes of the fiber optic laser light delivery system 8 and optical image collection system 
29. This ensures a full and uniform laser illumination of tissue 10 within the camera field-of-view. Depending on the 
blood flow range and sensitivity required by the measurement, fiber optic light delivery system 8 may consist of various 
combinations of single or bundled multimode or single mode, polarization maintaining, or non-polarization maintaining 
fiber optic cables. 

15 [0026] In the preferred embodiment of the instrument shown in FIGS 1 and 2, images of laser illuminated tissue are 
captured using much of the standard endoscope hardware including; imaging relay lens system 29 contained within 
endoscope shaft 2 for optical transfer of laser illuminated tissue images to camera focal plane, camera for electronic 
capture of laser illuminated tissue 1 , and electronic cable 12 for transferring video signals to video monitor system 5 
(1 st display). 

20 [0027] Modifications to the standard hardware for transfer of video signals include; video signal 1 -to-2 channel splitter 
13, for splitting channel 12 containing video signal output from camera 1 into 2 channels, 14 and 15, where channel 
14 feeds the camera video signal to a standard endoscope monitor 5 (1 st display) for the real-time, video feed of light- 
illuminated tissue used by endoscopic surgeons for maneuvering about the joint cavity and channel 15 for transferring 
camera video signal to a frame grabber card 1 6. 

25 [0028] The frame grabber card 1 6 digitizes analog video signals and digital data is transferred to a computer process- 
ing system 1 8, via digital, data-transfer bus 1 7. Digitized, laser-illuminated images are processed by computer process- 
ing system 18 into blood flow images using a speckle reduction and comparison method for generating images con- 
taining blood flow information, which includes the steps of: capturing a sequence of speckle images of laser illuminated 
tissue; comparing these speckle images to a reference image that is an image of the illuminated tissue where speckle 

30 has been reduced; and representing information obtained from this comparison in image format such that images are 
generated with a relation to blood flow. The speckle reduction and comparison method is described in more detail 
below starting in paragraph 36. Images are transferred to digital storage medium 20 via data transfer bus 19. Images 
are also transferred to monitor 22 (2 nd display) for real-time, immediate display of processed, blood flow images via 
data bus 21 . 

35 [0029] In an alternate embodiment of the method for image capture the standard endoscope camera is replaced by 
a high resolution, scientific quality digital camera with associated analog to digital conversion hardware. Digital output 
is then fed directly to the computer processing system 18 for processing, storage, and display of blood flow and laser 
speckle images. 

[0030] When an object surface has a roughness of less than one wavelength, the light field at the diffraction plane 

40 has a non-vanishing specular component superimposed with a fluctuating diffuse component. For measurements in 
tissue, the diffuse component originates from tissue scattering points below the surface and is the fluctuating light 
component leading to the speckle pattern. This is the light component that carries information of blood flow and, in 
cases when the tissue surface is very smooth, can be relatively weak compared to the specular component. 
[0031] Internal body tissue can be very smooth, having a high specular light component when illuminated. Therefore, 

45 it is effective for a blood perfusion imager to use a technique to eliminate or reduce the specular component in the 
measured light reflected back from the tissue surface. A further embodiment of the invention uses a polarization tech- 
nique to reduce specular reflection such that a polarized light source is used in conjunction with a polarization filter 
with the polarization axis aligned perpendicular to the polarization axis of the illuminating light. This filter can be placed 
at any location within the light receiving optics of the endoscopic imaging relay lens system 29. 

50 [0032] In the preferred embodiment of the method of light delivery, polarization of the light source can be achieved 
by attaching a polarization filter over the internal light guide exit port located at the distal tip of the endoscope. This 
filter must not cover the light receiving optics also located at the distal tip of the endoscope shaft. For most standard 
endoscopes, this requires a ring shaped design. Alternatively, a circular polarization filtering technique can be used 
with a polarization filter and quarter wave-plate located at the distal tip of the endoscopic device. 

55 [0033] In the alternate embodiment of the method of light delivery, a polarized laser source can be used with a single 
mode, polarization maintaining fiber optic for fiber optic cable 8, eliminating the need for a polarization filter placed 
over the source light. 

[0034] For some measurements, surface reflection does not present a significant problem or motion of the object 
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surface itself is desired. Therefore, this endoscopic "cross polarization" technique is optional and can be used to reduce 
surface reflections when sub-surface diffuse light is the desired scattered light component. 

[0035] For optical perfusion measurements using coherent light sources it may be necessary to reduce non-coherent 
background light from the measurement. The simplest method would be to remove all ambient light sources during a 
measurement. However during endoscopic surgery it may be difficult to remove all ambient surgical light while a blood 
flow measurement is performed. Therefore, a further embodiment of the invention uses a narrow band-pass filter, of 
3 -10 nm wavelength range, centered on the source wavelength and placed within the endoscopic imaging relay lens 
system 29 for effectively reducing ambient light during endoscopic blood flow measurement. 

[0036] Endoscopic blood flow applications include but are not limited to measurement in an abdominal cavity, a joint 
cavity, a pulmonary cavity, and a bladder. 

[0037] A speckle reduction and comparison method for generating relative blood flow images from laser speckle 
images will now be described. The blood flow algorithms described in this method can be used to measure motion of 
any object including and not limited to blood flow and bio-activity, where bioactivity applications include and are not 
limited to measuring the ripeness of fruit and vegetables, the vitality of live tissue and to ascertain the time of death 
for dead tissue. Various modifications could be made to the algorithms and mechanical methods used to perform a 
speckle reduced, reference image technique for generating blood flow images through the analysis of laser speckle 
images captured using a two dimensional detector with a finite integration time. 

[0038] This method can be used for any instrument using a two-dimensional photo-detector and focusing lens ar- 
rangement, endoscopic or otherwise. For example, an electronic camera attached to any focusing lens arrangement, 
including but not limited to: a horoscope lens system for measuring motion of mechanical parts within mechanical 
cavities, a microscope arrangement for measuring microscopic motion including microvascular blood flow, a video lens 
for measuring larger objects such as areas of exposed tissue including organs and body parts, a close focus zoom 
lens for measuring smaller objects such as regions of exposed tissue, an opthalmoscope for measuring retinal blood 
flow, and a zoom lens for measuring motion in distant objects. The same components shown in FIGS. 1 and 2 would 
be used, except that the endoscopic components would be replaced with the respective video and light guide systems 
mentioned here. In FIG. 2, the laser transmitter 6 would be coupled to a suitable light guiding arrangement for uniform 
illumination of the target 1 0 instead of fitting to the endoscopic light guide via lens 7, light guide 8 and optical or me- 
chanical switching connection 30. In FIG. 2 the endoscopic light collection system 29 and camera 1 would be replaced 
by a lens system appropriate for the desired application coupled to a standard electronic camera. 
[0039] Exemplary calculations used for generating a blood flow image that are disclosed in this patent require cap- 
turing an image (or sequence of images) of the tissue of interest while it is illuminated by coherent laser light. The 
coherent laser light produces a speckle structure in the image of the illuminated tissue. The speckle structures within 
the tissue image are a random interference pattern created by irregularities on and near the surface of the laser- 
illuminated tissue. These speckles appear across the tissue image and have a size given by: 

o = -\.2C\+M)XF, (1) 

where M is the magnification of the lens, A. is the wavelength of the laser light, and F is the camera F-stop number (A. 
E. Ennos, edited by J. C. Dainty, In: "Laser Speckle and Related Phenomena", Topics in Applied Physics. 9 , Springer 
Verlag, New York, 1984). The independent variables of the above formula are chosen to produce a speckle size on 
the order of the pixel size in the image. This is to ensure that the statistics performed on the image pixels is as close 
as possible to the statistics for the speckle theory. 

[0040] The speckle pattern produced by a stationary object remains static in time and contains a distinctive speckle 
structure. If the object contains several individual particles undergoing motion, such as red blood cells, then the phase 
difference between the interfering beams will change in time producing a dynamic speckle pattern. If an image of a 
dynamic speckle structure is captured over a finite integration time then several speckle patterns will become super- 
imposed over one another and the single, well-contrasted speckle pattern becomes destroyed. This relation between 
motion and the destruction of the speckle pattern over a finite integration time is the basis behind quantifying blood 
flow, where the extent of destruction depends on the level of flow for the moving red blood cells. Therefore, an image 
processing method for generating blood flow images as illustrated here requires a method for measuring the level of 
speckle destruction for each point across the tissue surface. This method uses a speckle reduction technique to gen- 
erate a speckle-free image of the illuminated tissue that can be used as a reference image to evaluate the speckle 
structure in the laser-illuminated image. The results of this comparison are then used to quantify motion or flow. 
[0041] A digital laser speckle image is composed of measured intensities l(x,y), where x and y represent the location 
of the pixel containing intensity, I. The resolution of the image is determined by the number of rows and columns. A 
typical video image contains 480 rows and 640 columns, although higher resolution cameras are available. The purpose 
of these calculations is to maintain the field of view and resolution of the captured video images in the generated blood 
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flow images. This allows the physician to make anatomical comparisons between the blood flow images and the con- 
tinuous endoscopic video feed. This is desirable in an endoscopic blood flow instrument. Subsequently, the calculations 
presented here use a high-resolution two-dimensional photo -detector and maintain the resolution in the generated 
blood flow images. This is different from the prior art, which is either limited to a low-resolution photo-detector (Konishi 
s and Fujii, 1 995; Tamaki, et al, 1 994; and US patents 51 63437 and 5240006) or use a high-resolution camera but lose 
the resolution in the generated speckle contrast images (Briers, et al 1999; Briers and Webster, 1995; Dunn et al, 
2001). Therefore, prior art LSI techniques result in image resolutions of around 100 x 100 pixels or less, a resolution 
far too low to be effective for clinical use. 

[0042] The exemplary calculations disclosed in this patent use a reference image and comparison technique to 
10 produce images of tissue blood flow. This reference image is generated by using a speckle reduction technique to 
reduce or eliminate the speckle within an image that is otherwise identical to the speckle image produced when tissue 
is illuminated by highly coherent laser radiation. The level of speckle destruction across a laser illuminated tissue 
surface can be quantified into a perfusion index linearly related to tissue blood flow using a software technique based 
on a statistical comparison between the pixels in the speckle reduced, reference image and speckle image captured 
'5 under coherent laser illumination. 

[0043] The reference image technique requires capturing or producing an image, where the speckle structure is 
removed or suitably reduced, l REF . This can be achieved either through a mechanical or software method. For example, 
mechanical methods for generating the reference image can be performed by illuminating the tissue with the same 
laser .light used to capture speckle contrast images but reducing the coherence of the source by vibrating the camera 
20 or multi-mode laser fiber during image capture. The amplitude of the vibration must be enough to destroy the speckle 
structure yet small enough not to destroy the sharpness of the image. Also, the period of the vibration must be several 
magnitudes smaller than the capture time so as to superimpose several speckle images of various phase differences 
during image capture. Thus, destroying the speckle structure on the imaged tissue. 

[0044] Another mechanical method is to illuminate the tissue with a light source of short coherence length. With the 
25 proper optical arrangement, the short coherence length source can be made to produce an image identical to the laser- 
illuminated tissue without the speckle structure. The short coherence length source intensity should illuminate the 
camera field of view in an identical way as the laser source and is normalized to produce the same detector output 
current as that produced by the laser-source when temporal or spatial speckle fluctuations are averaged out. One 
obvious example of a short coherence length source that can be used is the white light source used in standard en- 
30 doscopes. Another example would be to run the laser used to illuminate the object in a mode that would reduce the 
coherence length. This can be achieved by using a suitable voltage or current level in the laser driver. Another example 
would be to introduce a diffusing optical element into the laser source light path. 

[0045] A software method for producing the reference image is the preferred embodiment presented in this patent. 
[0046] An exemplary and preferred software method removes the speckle structure using a combined spatial smooth- 
35 ing and time-averaging calculation on each pixel within the captured speckle images: 



^(^) = ^|[X2 I + 1 )|(X2y + i)|'-»<^))]> <*> 

where i REF (x,y) represents the intensity of the (x,y) pixel in the reference image and 

'spw ( x <y) represents the intensity 

of the (x,y) pixel in the N" 1 speckle image of laser illuminated tissue for a captured sequence of N MAX images and / 
45 and j represent the boundaries for a chosen region of pixels surrounding the (x,y) pixel. 

[0047] Spatial smoothing is an image processing term used for performing an average calculation on each pixel (x, 
y) using its nearest neighbors. For example, the intensity of the (x,y) pixel within the original speckle image becomes 
replaced with the average intensity for all of the pixels surrounding the (x,y) pixel within a region defined by horizontal 
and vertical pixel dimensions of 2i+1 and 2j+1 . The size of the region should be several times larger than the dimensions 
50 of the speckle to ensure that the speckle spatial frequency will be sufficiently smoothed out in the reference image. 
[0048] For an object with a dynamic speckle pattern, such as living tissue, the speckle structure can be removed by 
performing a time averaging over a sequence of N MAX captured speckle images. The order and capture frequency of 
the speckle images within the sequence is unimportant, as long as the total exposure time for the captured sequence 
is several times larger than the decorrelation time of the dynamic speckle pattern; 

55 

Tseo = "max * T, (3) 



7 



EP 1 324 051 A1 



where T is the camera integration time and N MAX \s the number of images in the captured sequence. This will ensure 
a temporal smoothing of the time averaged speckle image generated from the sequence. For example, bioactivity 
decorrelates a speckle structure within several milliseconds, which is usually an order of magnitude greater than vas- 
cular blood flow. Thus, for living tissue, a captured sequence at standard video rates (30 frames per second) is usually 
sufficient for a good time average. 

[0049] Blood flow within a captured speckle image is quantified by performing a sum of difference calculation on the 
pixels in the captured speckle images / SPW and the corresponding pixels in the generated reference image, l REF 



'5 where l SD (x,y) is the (x,y) pixel in the sum of difference image, / SD , and N MAX \s the number of images in the captured 
sequence. 

[0050] Higher blood flow will result in a smaller value for / SD (x,y). When the blood flow reaches the maximum value 
measurable for the chosen camera integration time the speckle image will become completely destroyed and the sum 
of difference calculation (equation 4) will approach zero. Therefore the blood flow index has an inverse relation to the 
20 sum of difference value, / SD (x,y). This value must also be normalized with the total light intensity emitted from the 
tissue at each location across the image using the reference image, l REF (x,y). 
[0051] Therefore, the final blood flow image is given by; 



30 where l BF (x,y) represents the intensity of the (x,y) pixel in the blood flow image, C represents theorder of terms in the 
power series of highest order C MAX of the independent variable that is the ratio of l REF (x,y) and / SD (x,y), and A c is 
the constant coefficient for each term. 

[0052] The number of terms in the series and the constant coefficients for these terms will vary with each tissue and 
tissue blood flow range. This is to ensure the best possible linear relation between the blood flow image algorithm and 
35 tissue blood flow. For example, using only the zero and first order terms in the series are typical, however, the zero 
and second order terms, or zero, first, and second order terms could also be used if an empirical analysis shows this 
to provide a better linear blood flow relation. (It should be noted that the zero order term is used to remove the inherent 
instrument offset for tissue having no blood flow.) 

[0053] Equations 2, 4 and 5 represent a temporal and spatial comparison on a population of speckles for a given 

40 region within a reference image and a captured sequence of speckle images. The independent variables i, j, and N MAX 
are equation parameters chosen by the operator to reflect the desired quality of the blood flow calculations balanced 
with the quality of the image detail and temporal response of the instrument. For example, the statistics of the com- 
parison between the reference image and the speckle images is improved by increasing the number of speckles an- 
alyzed. A larger number of speckles are achieved by increasing i and j or N MAX . 

45 [0054] For a maximum detail in blood flow images i and j can be set to zero and a strict first order comparison of 
temporal speckles is performed. This would be used if the physician requires capturing a blood flow image at the highest 
possible detail to analyze tissue microvascular structure. A larger capture sequence requires a longer time between 
successive blood flow images but a reasonable blood flow image can be produced using a value of N MAX equal to 20. 
[0055] The methods for analyzing temporal speckles to generate blood flow images that exist in the prior art quantify 

so blood flow by measuring the differences in intensity of individual speckles in successive scans. This analysis is a time 
differentiated technique rather than our time integrated method and requires a high-speed capturing device that is 
limited to a lower resolution (Fujii, 1987; Fukuoka, 1999; Fukuoka, 1999: and US patents 4862894, 5163437 and 
5240006). These time differentiation methods can also lead to erroneous measurements because of the finite capture 
time imposed by a time integrating capturing device such as a CCD (Konishi and Fujii, 1995; Tamaki, et al, 1994). 

55 [0056] For a maximum blood flow video frame rate N MAX can be set to one and a strict first order comparison of the 
spatial speckle population is performed. This would be used if the physician wants to observe high frequency blood 
flow fluctuations within the tissue of interest. Larger values of i and j will also smooth out image details such as ana- 
tomical margins and boundaries but a reasonable blood flow image can be produced using a value of 2 or 3 for both 




(4) 
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iandj. 

[0057] Methods for analyzing spatial structures within laser speckle images that exist in prior art use a statistical 
contrast calculation on the speckle population within an analysis window that results in a considerable reduction in the 
resolution of the final blood flow image compared to that of the captured speckle image (Briers, 1999; Briers and 
Webster, 1995; Dunn et al, 2001). An exemplary method disclosed in this patent maintains the image pixel resolution 
and optical field of view between the laser speckle image and the generated blood flow image. In this way, a one to 
one pixel correlation between a live video feed of the speckle images and the blood flow image is possible. This is 
desirable for an effective clinical instrument designed for endoscopic applications. 

[0058] Prior art spatial analysis techniques are also limited to producing speckle contrast images that have a non- 
linear relation with tissue blood flow (Briers, 1999: Briers and Webster, 1995). 

[0059] The combined spatial and temporal comparison algorithm (equations 2, 4 and 5) with the user defined spatial/ 
temporal comparison parameters (/, /', and N MAX ) make the endoscopic instrument a more versatile clinical tool because 
the operator is free to decide on the desired resolution and the acceptable signal to noise ratios for any given physio- 
logical or clinical measurement. 

[0060] To improve the image quality and reduce noise, a software averaging of several blood flow images is available. 
This method can also be used to average out unwanted pulsatile frequencies that may be present in the flow if meas- 
urement of only low frequency blood flow change is desired. The average calculation is given by; 



(6) 



where l BFn (x,y) represents the intensity of the (x,y) pixel in the n' h blood flow image for a sequence of 

n MAX blood flow 

25 ■ images. 

[0061] In a clinical setting, relative motion between the patient and operator can occur creating what is known in the 
field of blood perfusion measurement as "motion artifact". Motion artifact will add to the destruction of speckle in the 
captured speckle images, giving a positive offset in the final blood flow images. This artifact can be eliminated by 
holding both the patient and instrument fixed during image capture. However, in a clinical setting this is difficult to 
30 achieve. Therefore, for an effective clinical LSI instrument, it is important to determine when motion artifact occurs 
during image capture and discard those images from blood flow calculations. 

[0062] An exemplary and preferred method for reducing motion artifact in flow or motion measurement is now pre- 
sented. 

[0063] FIG. 4 demonstrates an exemplary method disclosed in this patent to determine if patient/operator artifact 

35 motion occurs during capture of a speckle image. 

[0064] Motion artifact can be determined by analyzing stationary "target" areas within a captured speckle image. 
These stationary target areas should represent a well-contrasted speckle pattern and sometimes appear naturally 
within the image. For example, the camera field of view 26 normally contains an area containing various tissue types 
surrounding the specific tissue of interest 1 0. If one of these tissue types is a tissue known to contain minimal flow 24, 

40 such as cartilage, then the image region containing this tissue is specified as a target region. If a minimal flow target 
cannot be identified within the camera field of view then a special probe 25 containing a target cell 27 is placed within 
the camera field of view 26 next to the tissue of interest 10. The target cell comprises a flat painted metal surface of 
average surface roughness greater than one wavelength, producing a well-contrasted speckle pattern. 
[0065] Various methods can be used to analyze the speckle structure within the target region to determine if motion 

45 artifact occurs. These methods require determining an average value for the pixels within the defined region and eval- 
uating the minimum sum of difference value for each pixel and the associated average value to evaluate the speckle 
structure over the illuminated .target region. For a stationary target with zero flow the sum of difference value is at a 
maximum. This maximum depends on characteristics of the light source (polarization and modal quality) and tissue 
and must be determined in a calibration measurement that is performed when little or no motion is occurring. When 

so motion artifact occurs the speckle structure over the target becomes destroyed and the sum of difference value de- 
creases, approaching zero. 

[0066] The various motion artifact detection methods require 6 basic steps: 

1) Identifying a stationary target tissue 24 within the camera field of view or, in the absence of any stationary tissue, 
55 probe 25 containing the target cell 27 is placed within the camera field of view and identified within the image. 

2) Capturing a speckle image containing the tissue of interest and the stationary target. 

3) Defining the region of pixels within the image containing the target. 

4) Generating an average intensity value for the pixels within the defined region. 
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5) Determining the level of destruction in speckle structure as a result of relative motion by performing a sum of 
difference calculation on the defined region of pixels within the captured speckle image and the average intensity 
value for those pixels. 

6) Discarding speckle images where the sum of difference value falls below a specified minimum, where such 
minimum represents the tolerance value specified by the operator for motion artifact that destroys the speckle 
pattern within a given image. 

7) Determining a specified minimum value from a calibration procedure that consists of capturing a speckle image 
in the absence of motion and determining the sum of difference value for the target region and adjusting this value 
to include a specified tolerance range. 

[0067] Variations on generating the average intensity value and sum of difference calculation in the measurement 
and calibration procedure may be used. An average intensity value used in step 4 can be determined specifically for 
each pixel within the defined region by generating a reference image from the captured speckle image 




20 

where equation 2 was used with N MAX setto one. The sum of difference calculation in step 5 is then performed between 
the captured speckle image and the generated reference image for the pixels within the target region 



TARGET^ = ]T |/ 5P (x, y) - I ^ (x, y)\ , (8) 



where TARGET SD represents the sum of differences of the (x,y) pixels in the captured speckle image, / sp , and the 
generated reference image, l REF , within the target region and summation is performed over all pixels within the defined 
region. 

[0068] Alternatively, an average intensity value, l AVG , is determined for all pixels within the entire analysis region 



V-)2>^)' (9) 

^ number )~: h 

where l(x,y) is the intensity of pixel (x, y) contained within the region, number is the number of pixels within the defined 
region, and summation is performed over all pixels within the region. The sum of difference value in step 5 is then 
determined using the normalized sum of difference, REGION SD , of all pixels within the region and the calculated av- 
erage intensity value 



y)-iAw\- (io) 



[0069] It is sometimes difficult to ensure that motion does not occur during the calibration procedure for determining 
the minimum sum of difference value. Therefore, this process can be automated using a software routine that captures 
a sequence of speckle images. The sum of difference value within the target region of each speckle image can be 
calculated and the minimum or average value of the entire sequence can be used as the target minimum value for 
step 6. This minimum value can then be adjusted by adding a specified offset to enable a tolerance level to the blood 
flow measurement. This tolerance level depicts a balance between waiting a long time to capture a blood flow image 
with absolutely no motion and decreasing the length of time by allowing a moderate amount of motion artifact. 
[0070] Once the minimum value is calibrated and determined for a particular setup the value can' be stored in a look 
up table for future reference. Using this minimum target value, each speckle image that is captured is analyzed within 
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the specified target region to determine if the target sum of difference value falls below the minimum target value. 
When the target sum of difference value falls below the minimum, speckle images are immediately discarded before 
blood flow calculations are performed and a warning is given to the operator that motion artifact has occurred. 
[0071] This motion artifact feature should be used for endoscopic surgery as relative motion between the patient 

5 and surgeon can easily occur during capture of a laser speckle image. 

[0072] The noise reduction techniques in this instrument are unique from prior art and the specific methods are 
desirable for endoscopic applications in which some amount of patient/operator motion will inevitably be present. 
[0073] A person skilled in the art could make immaterial modifications to the invention described and claimed in this 
patent without departing from the essence of the invention. For example, new endoscopic systems use digital cameras, 

10 where digitization of CCD signals is performed within the camera system rather than on a frame grabber card. These 
digitized images are transferred along digital transfer cables rather than standard analog video signal cables and a 
direct connection to the computer processing system bus is possible circumventing the need for a frame grabber card. 
Also, we have presented only two of the possible methods of laser light delivery to the tissue. One method uses the 
endoscope's internal light guide, easily conforming to standard endoscopic procedures. The alternate method requires 

15 a custom light guide, external to the standard endoscopic shaft containing the relay lens system and internal light guide. 
This method requires minor modifications to the endoscopic probe but provides better quality blood flow measurements. 



Claims 

1 . An apparatus for the endoscopic measurement of motion within an enclosed cavity, comprising: 

an electro-magnetic radiation source; 

a guide for guiding electro-magnetic radiation from the electro-magnetic radiation source down an endoscopic 
device to illuminate a target containing motion within an enclosed cavity; 

an electro-magnetic radiation receiving system containing an optical arrangement for endoscopically relaying 
an image of the target onto an electro-magnetic radiation receiving element; 

a capture device for capturing the image of the target from the electro-magnetic radiation receiving element; 
and 

a processing device connected to the capture device for calculating motion information from the captured 
image. 

2. The apparatus of claim 1 , wherein the image of the target contains speckle structure information. 

3. The apparatus of claim 1 , wherein said processing device operates on the speckle structure information to generate 
images of motion. 

4. The apparatus of claim 1 , further comprising a display connected to one of the electro-magnetic radiation receiving 
system, the capture device and the processing device for the immediate visual display of two-dimensional color- 
coded images representing various regions containing different levels of motion. 

5. The apparatus of claim 1 , wherein said processing device uses a speckle reduction and comparison method for 
generating images containing blood flow information, where such method comprises the steps of: 

capturing a sequence of speckle images of laser illuminated tissue; 

comparing these speckle images to a reference image that is an image of the illuminated tissue where speckle 
has been reduced; and 

representing information obtained from this comparison in image format such that images are generated with 
a relation to blood flow. 

6. The apparatus of claim 5, in which the method for generating images containing blood flow information further 
comprises a method for capturing a reference image that is similar to the speckle images of laser illuminated tissue 
but with reduced speckle structure by illuminating the tissue with a light source of reduced coherence. 

7. The apparatus of claim 5, in which the method for generating images containing blood flow information further 
comprises a software method for reducing speckle within the captured speckle images of laser illuminated tissue, 
l SPN , to generate a reference image, l REF , where 
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and represents the intensity of the (x,y) pixel in the generated reference image and l SPN (x,y) represents the 
intensity of the (x,y) pixel in the N th speckle image of the captured sequence of N MAX speckle images. / and / 
represent the boundaries for a chosen region of pixels surrounding the (x,y) pixel. 

8. The apparatus of claim 5, in which the method for generating images containing blood flow information further 
comprises a software method for comparing the speckle images with the reference image, where such method 



determining the sum of difference between the pixels in the speckle images and the reference image using 
the calculation; 



where l REF (x,y), l SPN (x,y) and / SD (x,y) represent the intensity of the (x,y) pixel in the reference image, the 
N th speckle image and the generated sum of difference image, respectively. / and / represent the boundaries 
for a chosen region of pixels surrounding the (x,y) pixel; and 

generating a blood flow image from the reference and sum of difference images using the following calculation; 



where l BF (x,y) represents the intensity of the (x,y) pixel in the blood flow image, C represents the order of 
terms in the power series of highest order C MAX of the independent variable that is the ratio of l REF (x,y), and 
'so ( x <y)< and A c is the coefficient for each term. 

9. A method of detecting motion comprising the steps of: 

capturing a sequence of speckle images of a laser illuminated target; 

comparing the speckle images to a reference image that is an image of the illuminated target where speckle 
has been reduced; and 

representing information obtained from this comparison in image format such that images are generated with 
a relation to motion. 

10. The method of claim 9 further comprising capturing a reference image that is similar to the speckle images of the 
laser illuminated target but with reduced speckle structure by illuminating the target with a light source of reduced 



11. The method of claim 9 further comprising a software method for reducing speckle within the captured speckle 
images of a laser illuminated target, l SRN , to generate a reference image, / REF , where 



consists of: 





coherence. 
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and represents the intensity of the (x,y) pixel in the generated reference image and / SPW (x,y) represents the 
intensity of the (x,y) pixel in the N th speckle image of the captured sequence of N MAX speckle images. / and j 
represent the boundaries for a chosen region of pixels surrounding the (x,y) pixel. 

12. The method of claim 9. in which the method for generating images containing motion information further comprises 
a software method for comparing the speckle images with the reference image, where such method consists of: 

determining the sum of difference between the pixels in the speckle images and the reference image using 
the calculation; 



where l REF (x,y), l SPN (x,y) and / SD (x,y) represent the intensity of the (x,y) pixel in the reference image, the 
N th speckle image and the generated sum of difference image, respectively. ;' and j represent the boundaries 
for a chosen region of pixels surrounding the (x,y) pixel; and 

generating a motion image from the reference and sum of difference images using the following calculation; 



where l Motion (x,y) represents the intensity of the (x,y) pixel in the motion image, C represents the order of 
terms in the power series of highest order C MAX of the independent variable that is the ratio of l REF (x,y), and 
'so ( x <y)> and A c is lhe coefficient for each term. 

13. The apparatus of claim 5 in which the processing device is configured for eliminating motion artifact in the blood 
flow images by evaluating the speckle structure of laser light reflected from a stationary target located within the 
field of view of a captured speckle image. 

14. The apparatus of claim 5 in which the processing device is configured for eliminating motion artifact in blood flow 
images by: 1 ) using a stationary target containing a well-contrasted speckle pattern within the illuminated region, 
2) capturing a speckle image containing the tissue of interest and the stationary target, 3) defining a region of 
pixels within the image containing the stationary target, 4) determining the average pixel intensity, l AVG , for all 
pixels contained within the defined region 



JZ/(^), 



where l(x,y) is the intensity of pixel (x, y) contained within the region of pixels, "number" is the number of pixels 
within the region, and the summation is performed over all pixels within the region, 5) determining the normalized 
sum of difference, REGION SD , of all pixels within the region using the calculated average intensity 
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REGION SD = 



and, 6) discarding speckle images where the REGION SD value falls below a specified minimum, where such min- 
imum represents the tolerance value specified by the operator for motion artifact that destroys the speckle pattern 
within a given image. 

The method of claim 9 further comprising eliminating motion artifact in the motion images by evaluating the speckle 
structure of laser light reflected from a stationary target located within the field of view of a captured speckle image. 

The method of claim 9 further comprising eliminating motion artifact in motion images by: 1) using a stationary 
target containing a well-contrasted speckle pattern within the illuminated region, 2) capturing a speckle image 
containing the target of interest and the stationary target, 3) defining a region of pixels within the image containing 
the stationary target, 4) determining the average pixel intensity, l AVG , for all pixels contained within the defined 
region 



where l(x,y) is the intensity of pixel (x, y) contained within the region of pixels, "number" is the number of pixels 
within the region, and the summation is performed over all pixels within the region, 5) determining the normalized 
sum of difference, REGION SD , of all pixels within the region using the calculated average intensity 



and, 6) discarding speckle images where the REGION SD value falls below a specified minimum, where such min- 
imum represents the tolerance value specified by the operator for motion artifact that destroys the speckle pattern 
within a given image. 

The apparatus of claim 1 , further comprising a display connected to one of the electro-magnetic radiation receiving 
system, the capture device and the processing device for the immediate visual display of captured two-dimensional 
images of the illuminated target. 

The apparatus in claim 1 , used for the detection of motion, where the motion comprises the flow of blood within 
an illuminated tissue enclosed within a body cavity. 

The method of claim 9, where the motion comprises the flow of blood within an illuminated tissue forming the laser 
illuminated target. 
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